feat(revise): verifiable evidence: quote and spot checks, one verdict per line, who ran a review - #122

Closes #88

Origin: #88, tracked in #89. In the reference revision, much of the report's evidence reached the group unchecked:

• proposals carried an analyst's paraphrase instead of a person's words;
• one probe line gave two verdicts and was counted as a break;
• 42 findings named no one as having run the review;
• the write-up had eight figures that nobody re-derived.

Sandbox: Sonnet, --check, against the sandbox's widgets repository with synthetic data.

• revise-prepare-evidence: 7 of 7 runs pass across three commits, 2 of 2 on the final one. Each run flagged the planted wrong item (slice-1#3) and put the spot check: 5 items · 1 wrong · error rate 20% line in the Head. Each also had quote_check.py refuse the altered quote, recopied the comment's words, and reran the check to a pass.
• source-reader-page-excerpt: 2 of 2 runs return verbatim Excerpt: lines and say the version is not in the file. The first batch was marked as failing because the scenario's regex was double-escaped; the saved outputs pass the corrected regex.

Canary: the next /lasp-dev:revise prepare.

• Its report's Head should carry a spot check: line, and .review/revision/spot-check.md should hold ten lines.
• quote_check.py should run before the commit question, and pass.
• Any record harvested after the merge should have reviewer: on every finding.

The failures to look for:

• a said: line whose words aren't in the comment it cites;
• a Head figure that no script printed.

What changes

Every kind of evidence the revision report rests on is now checked by a script, or by a reader who didn't produce it:

Files:

• Scripts:
  • scripts/quote_check.py, new;
  • scripts/spot_check.py, new;
  • scripts/window_cost.py, new;
  • scripts/block_tally.py;
  • scripts/record.py.
• Skills and agents:
  • skills/revise/prepare.md (steps 1, 2 and 10);
  • outline.md (the Head line; every quote is a said: line);
  • context-refresh.md;
  • SKILL.md;
  • skills/onboard/first-rules.md;
  • agents/source-reader.md;
  • agents/diff-probe.md.
• Docs:
  • docs/contracts.md: the record v15, analysis blocks v2 and sweep blocks v3, plus three new sections: "The report's quotes", "The spot check" and "The window's cost";
  • docs/design.md;
  • the README's .review/ table.
• Tests:
  • tests/test_quote_check.py, tests/test_spot_check.py and tests/test_window_cost.py, new;
  • test_block_tally.py, test_record.py and test_sandbox.py;
  • the fixture pr-0042/expected.yaml, regenerated (contract line and reviewer: lines only).
• Sandbox: the scenarios revise-prepare-evidence and source-reader-page-excerpt, and the fixture sandbox/fixture/evidence/, with synthetic comments only.
• Versions: revise 2.25.0 and onboard 0.6.7.

Not changed: the guidance items and guidance_check.py, which already checked quotes and schema; pr-review, whose attribution line already names who posted; ratchet_counts.py; capture.py.

Review

Three rounds, each by a fresh adversarial reviewer:

• Round 1: 3 blocking findings, all fixed.
  • Step 10's figures rule contradicted the Head's cost figures. The fix is window_cost.py.
  • The comment lists of earlier windows were never given to the check. The fix adds the capture directories.
  • A nested - said: line was skipped instead of checked. Such a line is now refused.
• Round 2: 3 blocking findings, all fixed.
  • window_cost.py refused a pull request that had no record. It now reads body.md.
  • A quote could go unchecked just because its pull request's list was not given. A quote is now unchecked only when the ledger's held candidates carry it, and matching is by repository and number.
  • Rates were not tied to the counts they came from. A rate is now written with its counts.
• Round 3: no blocking findings. Its two should-fix findings and one nit were fixed: held quotes match by words and URL, however the ledger lays them out; a missing threads directory is refused; a labelled second verdict, breaks: or breaks —, is caught.

Checks on the final commit: the unit suite (1,255 tests), the four shell tests, check_static.py and codespell all pass.

Decisions for the maintainer

• Record contract 14 → 15, approved before the change. reviewer is read from the finding's poster rather than parsed from the attribution line, because the two are the same account. Records of earlier versions have none, and nothing reads the field yet.
• Line forms, not JSON, for the probe and analyst blocks, approved before the change. A fixed line form that a script parses already works as a schema. Refusing a second verdict closes the diff-probe: turn limit too low, and a verdict that contradicts itself #50 gap without adding a format.
• Who spot-checks: the driver, not a new agent. The driver didn't extract the items, so it has no stake in them, and a new agent would cost a launch for ten reads. The seed is the window, so a rerun checks the same ten items.
• A held candidate's quote can go unchecked. It is unchecked only when no comment list given holds its pull request and the ledger's held candidates carry the same words and URL. The count shows in the Head. Its words were checked in the window that held it.
• Only said: lines are checked. The outline now says every quote in the report is a said: line, so a person's words given any other way count as paraphrase. Proposals that rest only on agree rates (Dial rewordings, expiries) carry no quote and aren't required to.

🤖 Generated with Claude Code. AI assistance level: 5/5 (not read by a person).